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DETAILED ACTION 

1 . The Examiner notes that claim 28 has been cancelled and claim 29 has been 
added via the amendment filed 5/21/2008. 

Response to Arguments 

2. Applicant's arguments filed 5/21/2008 have been fully considered but they are 
not persuasive. 

3. Re claim 16, the Applicant contends that steps 302-306 utilize histograms 
representing visual content, not any difference of the visual content. While the 
Examiner agrees that the cited steps do not reflect differences between frames, the 
Examiner respectfully disagrees with the Applicant's assertion that Toklu fails to teach 
or suggest this aspect. Toklu discloses that camera motion information between 
consecutive frames of the segment is estimated (Toklu: Fig. 2B, step 212; column 8, 
lines 3-8). Therefore, the camera motion corresponds to the difference between two 
frames. Toklu further discloses sequentially determining a cumulative sum of motion in 
the frame (Toklu: Fig. 2B, step 219; column 10, lines 23-47), wherein the amount of 
motion corresponds to a rate of change in the frame. 

4. Re claim 1 , the Applicant contends that Toklu fails to teach or suggest that the 
rate of change calculations are repeated for each of the frames in the video sequence. 
However, the Examiner respectfully disagrees. Toklu discloses that the camera motion 
is summed for each frame (Toklu: column 10, lines 32-35) and that camera motion 
information between consecutive frames of the segment is estimated (Toklu: Fig. 2B, 
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step 212; column 8, lines 3-8). Therefore, the reference changes corresponding to each 
frame of the sequence. 



Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

6. Claims 1-2, 14-17, 23-24, and 29 are rejected under 35 U.S.C. 102(a) as 
being anticipated by Toklu et al. (US 6549643 B1). 

Re claim 1, Tolku discloses a method of identifying a plurality of key video 
frames in a sequence of image frames, each of said sequence of image frames 
containing a plurality of pixels, each of said plurality of pixels corresponding to a 
corresponding point of an area based on which said sequence of image frames are 
generated, said method comprising: determining a rate of change of visual content of 
each current frame from a corresponding reference frame, each of said current frame 
and said reference frame being comprised in said sequence of image frames (Toklu: 
Fig. 2B, step 212), wherein said rate of change represents a difference of a first value 
and a second value, said first value representing a change of visual content of a current 
frame compared to a first frame, said second value representing a change of visual 
content of said first frame compared to a second frame, wherein said second frame is a 
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reference frame for said first frame and said first frame is a reference frame for said 
current frame (Toklu: column 8, line 15, through column 9, line 9, calculating an amount 
of motion between two frames; Toklu: column 9, lines 13-31 and Eqn. 7, motion activity 
found is then used to generate a motion activity (MA) curve; Toklu: Fig. 3, the potential 
keyframes undergo an elimination process; Toklu: column 13, lines 14-29, the 
histograms for candidate frames are compared with the histogram for a reference 
frame, wherein the motion activity for the candidate frame represents a first value, and 
the reference frame represents a second value, which are compared for the purpose of 
determining key frames); and selecting said current frame as a corresponding one of a 
set of potential video frames (Toklu: Fig. 2B, step 223) if said rate exceeds a 
corresponding first threshold value (Toklu: Fig. 2B, steps 220-222), wherein said 
plurality of key video frames are selected from said set of potential video frames (Toklu: 
Fig. 2A, step 209 refines key frame results from steps 218, 223, and 225 of Fig. 2B), 
wherein said determining and said selecting are repeated for each of said sequence of 
image frames as said current frame to form said set of potential video frames (Toklu: 
column 10, lines 54-57), wherein each of the respective first frame and the respective 
second frame are different for different ones of the current frame in said sequence of 
image frames (Toklu: Fig. 2B, step 212; column 8, lines 3-8, camera motion information 
between consecutive frames of the segment is estimated; column 10, lines 32-35, 
camera motion is summed for each frame). 

Re claim 2, Tolku discloses a majority of the features of claim 2, as discussed 
above in claim 1, and Toklu additionally discloses determining a displacement 



Application/Control Number: 10/708,905 Page 5 

Art Unit: 2621 

magnitude of each moved pixel of said current frame compared to the position in said 
first frame and of said first frame compared to the position in said second frame (Toklu: 
column 8, line 15, through column 9, line 9, calculating an amount of motion between 
two frames); and computing a first representative magnitude of said displacement 
magnitude for said moved pixels of said current frame compared to said first frame, and 
a second representative magnitude of said displacement magnitude for said moved 
pixels of said first frame compared to said second frame (Toklu: column 8, line 15, 
through column 9, line 9, calculating an amount of motion between two frames; Toklu: 
column 9, lines 13-31 and Eqn. 7, motion activity found is then used to generate a 
motion activity (MA) curve), wherein said first value and said second value respectively 
equal said first representative magnitude and said second representative magnitude 
such that said rate is computed as a difference of said first representative magnitude 
and said second representative magnitude (Toklu: Fig. 3, the potential keyframes 
undergo an elimination process; Toklu: column 13, lines 14-29, the histograms for 
candidate frames are compared with the histogram for a reference frame, wherein the 
motion activity for the candidate frame represents a first value, and the reference frame 
represents a second value, which are compared for the purpose of determining key 
frames). 

Re claim 14, Toklu discloses that the corresponding first frame and the 
corresponding second frame are selected at a same respective relative position in 
comparison to the position of the current frame such that each of the respective first 
frame and the respective second frame are different for different current frame (Toklu: 
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Fig. 2B, step 212; column 8, lines 3-8, camera motion information between consecutive 
frames of the segment is estimated; column 10, lines 32-35, camera motion is summed 
for each frame). 

Re claim 15, Tolku discloses that said current frame, said first frame and said 
second frame are in consecutive successive positions in said sequence of frames 
(Toklu: Fig. 2B, step 212; column 8, lines 3-8, camera motion information between 
consecutive frames of the segment is estimated; column 10, lines 32-35, camera motion 
is summed for each frame). 

Re claim 16, arguments analogous to those presented for claim 1 are applicable 
to claim 16, and, therefore, claim 16 has been analyzed and rejected with respect to 
claim 1 above. 

Claim 17 has been analyzed and rejected with respect to claim 2 above. 

Re claim 23, arguments analogous to those presented for claim 1 are applicable 
to claim 23, and, therefore, claim 23 has been analyzed and rejected with respect to 
claim 1 above. 

Claim 24 has been analyzed and rejected with respect to claim 2 above. 

Re claim 29, Toklu discloses a method of identifying a plurality of key video 
frames in a sequence of image frames, each of said sequence of image frames 
containing a plurality of pixels, each of said plurality of pixels corresponding to a 
corresponding point of an area based on which said sequence of image frames are 
generated, said method comprising: receiving said sequence of frames of a same 
scene/area of interest according to a sequential order (Toklu: Fig. 2A, step 200); 
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choosing one of said sequence of image frames as a current frame, a first flame being 
before said current flame and a second flame being before said first flame according to 
said sequential order, said first flame being at a first relative position in relation to said 
first flame in said sequential order and said second flame being at a second relative 
position in relation to said first flame in said sequential order (Toklu: Fig. 2B, step 212; 
column 8, lines 3-8, camera motion information between consecutive frames of the 
segment is estimated; column 10, lines 32-35, camera motion is summed for each 
frame); calculating a difference of a first value and a second value, said first value 
representing a measure of a change of visual content of said current flame compared to 
said first frame, and said second value representing a measure of a change of visual 
content of said first flame compared to a second frame (Toklu: Fig. 2B, step 21 2); 
selecting said current flame as a corresponding one of said plurality of key video frames 
if said difference exceeds a first threshold value and first value exceeds a second 
threshold value (Toklu: Fig. 2B, steps 220-223; Fig. 2A, step 209; Fig. 4, steps 405- 
407); and repeating said calculating and said selecting after choosing each of said 
sequence of image frames as said current frame to form said plurality of key video 
frames (Toklu: Fig. 2B, step 212, camera motion estimated between consecutive frames 
for each segment; Fig. 4, step 409, elimination process repeated for each segment), 
wherein each of the respective first frame and the respective second frame are different 
for different ones of the current frames, and are respectively determined based on the 
same first relative position and said second relative position in reference to the 
corresponding current frame (Toklu: Fig. 2B, step 212; column 8, lines 3-8, camera 
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motion information between consecutive frames of the segment is estimated; column 
10, lines 32-35, camera motion is summed for each frame). 

7. Claims 3-4, 6, 9-11, 13, 18-19, 21, 25-26, and 28 are rejected Under 35 U.S.C. 
103(a) as being unpatentable over Tolku et al. (US 6549643 B1) in view of Zhang et 
al. (US 7027513 B2). 

Re claim 3, Toklu discloses a majority of the features of claim 3, as discussed 
above in claim 2 but Toklu does not explicitly state that said first representative 
magnitude for said current frame equals an average of motion energy vector 
magnitudes of said moved pixels of said current frame in comparison with 
corresponding pixels of said first frame. However, Zhang discloses a method for 
extracting key frames from video using a triangle model of motion based on perceived 
motion energy, where the average magnitude of the motion vectors for a particular 
frame is calculated (Zhang: column 9, lines 25-38), and the average magnitude is used 
to calculate the perceived motion energy (Zhang: column 9, lines 39-49). Since both 
Tolku and Zhang relate to selecting key video frames based on motion analysis, one of 
ordinary skill in the art at the time of the invention would have found it obvious to 
combine the triangle method of Zhang with the histogram analysis of Tolku in order to 
provide a uniform and consistent selection method that yields a proper number of key 
frames that are most representative of the video sequence content (Zhang: column 5, 
lines 34-37). The combined method of Tolku and Zhang has all of the features of claim 
3. 
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Re claim 4, the combined method of Tolku and Zhang discloses a majority of the 
features of claim 4, as discussed above in claim 3. Additionally, Toklu discloses that 
said first threshold value is the same for all of said current frames (Toklu: Fig. 2B, when 
the cumulative motion in steps 220, 221 , and 222 exceeds a threshold, the frame is 
selected as a key frame in step 223), said selecting further comprises: including said 
current frame is in said set of potential video frames only if said first representative 
magnitude exceeds a second threshold (Toklu: Fig. 2A, step 209 eliminates 
substantially similar frames; column 7, lines 56-65, elimination methods are shown in 
Figs. 3 and 4; Fig. 4, step 405, indicates a difference threshold); and including only 
those of said set of potential video frames, which exceed said first threshold, in said 
plurality of key video frames (Toklu: Fig. 4, steps 405 and 406, if the motion is not 
significant, the candidate frame is eliminated). 

Re claim 6, the combined method of Tolku and Zhang discloses identifying a 
plurality of active pixels in said current frame, wherein; a pixel is considered an active 
pixel if a corresponding displacement magnitude is outside of a range, wherein only said 
plurality of active pixels are used by said computing (Toklu: Fig. 2C, step 231). 

Re claim 9, the combined method of Tolku and Zhang discloses enabling a user 
to specify one of a plurality of key video frames, wherein said plurality of key video 
frames are selected by said selecting (Zhang: Fig. 1, element 120; column 5, lines 43- 
44, indicate that the user may use the key frames to select the desired section of the 
video for display); and displaying said specified one of said plurality of key video frames 
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(Zhang: Fig. 1 , element 1 20; column 5, -lines 43-44, indicate that the user may use the 
key frames to select the desired section of the video for display). 

Re claim 10, the combined method of Tolku and Zhang discloses displaying a 
prior key video frame and a next key video frame in relation to said specified one of said 
plurality of key video frames, wherein said prior key video frame and said next key video 
frame are comprised in said plurality of key video frames (Zhang: Fig. 1 , element 1 20, 
key frames are displayed in a sequence). 

Re claim 11, the combined method of Tolku and Zhang discloses generating a 
display indicating the manner in which said plurality of key video frames are 
interspersed in said sequence of image frames, wherein said enabling is based on said 
display (Zhang: Fig. 12, the key frames are shown along with their corresponding frame 
numbers). 

Re claim 13, the combined method of Tolku and Zhang discloses generating a 
display listing said plurality of key video frames, wherein said enabling is based on said 
display (Zhang: Fig. 1, element 120; column 5, lines 43-44). 

Claim 18 has been analyzed and rejected with respect to claim 3 above. 

Claim 19 has been analyzed and rejected with respect to claim 4 above. 

Claim 21 has been analyzed and rejected with respect to claim 6 above. 

Claim 25 has been analyzed and rejected with respect to claim 3 above. 

Claim 26 has been analyzed and rejected with respect to claim 4 above. 
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8. Claims 5, 7, 8, 20, 22, and 27 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Tolku et al. (US 6549643 B1) in view of Zhang et al. (US 
7027513 B2) as applied to claims 3-4, 6, 9-11, 13, 18-19, 21, 25-26, and 28 above, 
and further in view of Ma et al. (US 20040088723 A1). 

Re claim 5, the combined method of Tolku and Zhang discloses a majority of the 
features of claim 5, as discussed above in claims 1-4, but does not specifically disclose 
that the first threshold and the second threshold are adjusted dynamically to ensure that 
a desired number of frames are selected as key video frames in a specified duration. 
However, Ma discloses a method for generating a video summary, where a binarization 
threshold is estimated in an adaptive manner (Ma: paragraph [0081]) when analyzing a 
video sequence for selecting key frames. Since Tolku, Zhang, and Ma relate to 
selecting key frames in a video sequence, one of ordinary skill in the art at the time of 
the invention would have found it obvious to combine the attention modeling of Ma with 
the combined key frame selection method of Tolku and Zhang in order to create a 
representative video summary consisting of very short video clips, which contain the 
video immediately preceding and immediately following key frames that have been 
selected (Ma: Fig. 20). The combined method of Tolku, Zhang, and Ma has all of the 
features of claim 5. 

Re claim 7, the combined method of Tolku, Zhang, and Ma discloses that said 
range set by a distance of two times the variance from the mean of a distribution (Ma: 
paragraphs [0081]-[0082], the threshold is a function of the variance). 
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Re claim 8, the combined method of Tolku, Zhang, and Ma discloses that said 
representative magnitude comprises an average of said active pixels (Ma: equation (12) 
and paragraph [0084]). 

Claim 20 has been analyzed and rejected with respect to claim 5 above. 

Claim 22 has been analyzed and rejected with respect to claim 7 above. 

Claim 27 has been analyzed .and rejected with respect to claim 5 above. 

9. Claim 12 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Tolku et al. (US 6549643 B1) in view of Zhang et al. (US 7027513 B2) as applied to 
claims 3-4, 6, 9-11, 13, 18-19, 21, 25-26, and 28 above, and further in view of Sull et 
al. (US 20060064716 A1). 

Re claim 12, the combined method of Tolku and Zhang discloses a majority of 
the features of claim 12, as discussed above in claims 1 -4, 6, and 9-1 1 , but does not 
specifically disclose that the display comprises a pie chart. However, Sull discloses 
techniques for navigating multiple video streams, where textual/visual information, such 
as a pie chart, may be displayed along with poster-thumbnails on the video selection 
screen for the user interface (Sull: paragraph [0307]). Since Tolku, Zhang, and Sull all 
relate to representative images for video sequences, one of ordinary skill in the art at 
the time of the invention would have found it obvious to combine the textual information 
of Sull with the keyframe selection of the combined method of Tolku and Zhang in order 
to provide the user with more information, such as date and time of broadcast (Sull: 
paragraph [0307]), for improving the user's ability to quickly find the desired video 



Application/Control Number: 10/708,905 Page 13 

Art Unit: 2621 

segment (Zhang: column 5, lines 44-45). The combined method of Tolku, Zhang, and 
Sull has all of the features of claim 1 2. 

Conclusion 

10. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure: 

a. Feature based hierarchical video segmentation; Bozdagi et al. (US 
6493042 B1 ) 

b. Method of selecting key-frames from a video sequence; Wilf et al. (US 
7184100 B1) 

c. System for automatic video segmentation and key frame extraction for 
video sequences having both sharp and gradual transitions; Zhang et al. (US 
5635982 A) 

Contact 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHRISTOPHER FINDLEY whose telephone number is 
(571)270-1199. The examiner can normally be reached on Monday through Friday, 
8:30AM-5:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Marsha D. Banks-Harold can be reached on 571-272-7905. The fax phone 
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number for the organization where this application or proceeding is assigned is 571- 
273-8300. 
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Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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